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ABSTRACT 

A research projoct sought to develop ineans of 
measuring children's attention to instructional films. Groups of five 
educable mentally handicapped (I!MH> students watched films; observers 
noted their attention or inattention to the screen at five-second 
intervals. Data for the groups were then compiled via time series 
analysis procedures to yield a composite index of viewer attention 
for the given intervals. The results showed that the younger students 
exhibited greater variances in attention, that there was a positive 
correlation between attention and content learned, and that media 
presentations could be grouped and ranked with respect to their 
attention-holding power. It was concluded that the observation 
technique and the time series analysis method were useful means which 
would /permit researchers to; 1) discriminate among segments of a film 
with flifferent attention levels; 2) analyze content across and 
between attention levels; and 3) establish attention files for groups 
of syudents, /subject matter areas, and media presentations. (PB) 
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ATTENTION PROFILE: 
EVALUATION OF INSTRUCTIONAL MEDIA 

Thomas E. Grayson 
ABSTRACT 

An Index of the relative entertainment appeal of Instructional 
films Is obtained by measuring a child's Interest by continuously 
recording his visual orientation toward or away from the media screen 
during presentation, Graphing the fluctuations In audience Interest 
for a particular film permits analysis of the film from moment to 
moment to discover those elements which were most compelling of 
attention and those which failed to hold the interest of the sample 
audience. 

Observer reliability for gathering attention profile data has 
been established and a methodology for time series data analysis has 
been emp toyed . 
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SPECIAL REPORT No. 72 I 3 
COMPUTER-BASED PROJECT for the EVALUATION 
of MEDIA for the HANO ICAPPED 

T I t I 6 ! ATTENTION PROFILE J EVALUATION OF INSTRUCTIONAL MEDIA 
BYi Thomas E* Grayson 

BACKGROUND 

The Computer Based Project for the Evaluation of Media for the Handicapped, 
based on contract KOEC-9-423617-4357 (616) between the Syracuse (N« Y. ) City School 
District and the >:edia Services and Captioned Films Branch, Bureau of Education 
for the Handicapped (United States Office of Education) for the five year period 
July 1, 1969 through June 30, 1974. The major goal is to improve the instruction 
of handicapped children through the development and use of an evaluation system 
to measure the instructional effectiveness of films and other materials with 
educablo mentally handicapped (EMH) children, in-service training and media support 
for special teachers, and studies related to the evaluation process and the 
populations used* 

The Project has concentrated on the 600 films and 200 filmstrips from the 
Media Services and Captioned Films (BEH - USOE) depository; however, specific 
packages from Project LIFE, various elementary math curricula, and selected 
programs from children's TV Workshop have also been evaluated. The evaluation 
model used requires that* 1) objectives of materials be specified and written; 

2) instruments be corr*tructed to test and measure effectiveness? andi 3) children 
be the major sources of evaluation information* A nurrber of instruments and 
methodologies are employed in the gathering of cognitive and affective data from 
900 EMH children and 80 special teachers to make the effectiveness decisions. 
Over half of the EMH population can neither read or write; therefore, a unique 
Student Response System (SRS) iu employed, consisting of a twenty station G.E.- 
1000 SRS which can be operated in a group or individual recording mode and is 
connected to a remote computer system. The computer capabilities consist of 
remote telephone connections to the Rome (N.Y,) Air Development Command, the 
Honeywell time-shared network, and the Schenectady (N.Y.) G E Research and 
Development Center; and batch mode capabilities of the Syracuse City Schools, 
Syracuse University, and various commercial sources. 

In-service and media support activities provide on-the-job training for 
teachers, teacher aides, equipment, and materials to tha special teachers in 
the city schools. The research activities have centered around investigations 
and special problems related to the development of the evaluation model, The 
four major areas considered arei 1) testing effects, 2) captioning effects, 

3) special student characteristics; and, 4) evaluation procedures validation. 

Documentation of the major activities appear in the five annual reports 
and the 600 evaluations prepared on materials used. Staff members ;.ere encouraged 
to prepare special reports and the attached paper is one of these. The opinions 
expressed in this publication do not necessarily reflect the position or policy 
of the Computer Based Project, the United states Office of Education) or the 
Syracuse City School District, and no official endorsement by any of the agencies 
should be inferred. 



Tho Computer Based Project measures a child's 'Interest' In a 
material by continuously recording his visual orientation toward or 
away from the movie screen during the film presentation. This results 
In an Index of the relative appeal ('reward 1 ) of any given f llm« 
Graphing the fluctuations of student attend I ng-behavlor permits the 
researcher and others to analyze the film from moment to moment to 
discover those elemsnts which most attract attention and those which 
fall to hold Interest of the sample audience (Palmar 1963). 

Films are a one-way medium; therefore, when a child views a film, 
It is clearly an Individualized experience, If there Is feedback and 
reinforcement added, It Is assumed to be Instrumental In learning. 
Furthermore, any verbal or visual medium which Induces the child to 
attend closely to spoken sounds or visuals can be a useful Instrument 
for training In discrimination learning, as well as for other learning 
tasks. 

Attention can be defined by a response or set of responses In a 
specified stimulus condition: the measure of the response Is usually 
continuous, Immediate, and hopefully objective and sensitive (Llnds)ey, 
1962), 

Measures of attention are based upon some general assumptions 
related to the gaining of knowledge (Information) from engaging In some 
form of activity. It is generally assumed that : 



L Information gained has high correlation with the degree to 
which one attends, 

2. The more one attends, the greater the learning potential, 

3. A decreased amount of attention to a stimulus will result 
In a decreased amount of learning. 

When considering visual attention as the primary measure of 
Interest In a film, book, T.V., fllmstrlp, etc., It Is assumed that 
comprehension Is greater when attending than when not attending, 

Palmer (196tf) developed a procedure for collecting and charting 
attention behavior of pre-schoolers while viewing television, The re- 
sults of these charts were used to determine the effectiveness of 
program sequences which would be Included In the final broadcast 
(Sesame Street 1963). Palmer's basic procedure for obtaining attention 
data has been adopted as part of the media evaluation procedure used 
at the Computer Based Project, 
PROCEDURE: 

Basically the procedure Is as follows: 

The dependent variable being measured Is the number of students 
v/lth eyes on screen. The Independent variable Is a pre~selected film to 
be evaluated at 5 second Intervals, The attention response Is defined 
as eyes focused on, or directed toward, the screen, 

Two dimensions are considered and collected simultaneously, i«e. f 
time samples and the number of subjects, After some experimentation 
with the length of time Intervals and subject sample sizes observed, a 



5 second t'ne Interval and samples consisting of five randomly 
selected students were used for data gathering situations. 

The attention response Is recorded by two observers positioned 
In front of a grou^ of students so as to clearly see 5 subjects 1 eyes 
as they view the media. When the student appears to be looking In the 
direction of the screen during the film presentation, an attention 
response !s recorded. Recording begins with the presentation of audio 
or visual stimuli from the film. A record Is made of the number of the 
selected 5 subjects whose eyes are fixed on the media Image at each 5 
second Interval on a form such as shown In Figure I. After several 
showings, samples of 5 subjects who saw the same media are compiled Into 
a time series tine graph Indicating the frequency of subjects watching 
at each time Interval. This composite frequency graph yields a visual 
representation of the attention behavior exhibited during the length of 
the media presentation 

The results reported from the data are used to: 1) locate and 
Identify areas (segments) of films where test I tern d I screpencles appear, 
2) analyze content area at different attention levels, and 3) establish 
attention files for various classes of children and for subject matter 
areas for a specific piece of media. 

ATTENTION OBSERVATION TECHNIQUE 
MATERIALS AN D EQUIPMENT 

1. Media (film, fllmstrip, T.V., etc.) 

2. Projector and screen. 



3. Recording data form and pencil (see appendix 1). 

k* Stop watch. 

5. Observation chair, 

The observation technique consists of recording the frequency 
of a given number of children watching a visual presentation on a time 
Interval basis so that comparisons can be made between the observed 
level of attention and the visual experience that was presented, Some 
major considerations In the technique are; 
SIZE O F SAM PLE 

A minimum of 5 children are to be randomly selected out of the 
audience by each observer. The sample should be easily viewed 
from the observer's position, and different samples should be 
selected when there Is more than one observer present during a 
showing • 

TIHF JNTERVAL j 

Recordings of the frequency of "eyes on screen* within a sample Is 
made at 5 second Intervals, Le., as the sweep hand of the stop 
watch passes each 5 second number (5, 10, 15, 60). The stop 
watch Is started with the presentation of either audio or visual 
stlmut 1 from the f 1 tm. 

ATTENTIO N RESPONSE 

When the eyes of the child are judged to be fixed on the screen 
at the moment of observation for the time Interval, the child Is 
regarded as attending, 



ATTENT [OH 03SERVER RELIABILITY 
A recent study shows: 

1. A bias to tho correlation value of r which Indicates 
that heavy consideration of a correlation coefficient 
as an Indicator of I nterobserver reliability should not 
be given, 

2. A graphic representation of data showing agreement, 
disagreement, direction, and frequency Is a useful tool 
for determining Interobserver reliability. 

INTE RRUPTIONS 

There are situations where a member of the sample will be Inter- 
rupted or disturbed, consequently affecting his attending behavior, 

a. If a subject leaves the audience, select another subject 
as a replacement and continue observing. 

b. If a subject Is Interrupted by a teacher or an observer, 
select another subject at the tine of the Interruption. 

c. If a selected subject Is not attending to the media 
during Its showing, keep the subject In sample and record 
as not attending , 

d. If there are equipment malfunctions, stop observing and 
take corrective action. Return to observing, noting the 
minute and second on the stop watch, and record the next 
observation In the box for that minute of time. 
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TjM£ SEfilES ANALYSIS OF ATTENTI ON DATA 

Spelgel and Murry (1961) have defined a time series as a set of 
observations taken at specified times, usually at equal Intervals. 
When the Independent variable x Is time, the data (dependent variable 
y) shows the values of y at various time Intervals. Mathematically, a 
time series Is defined by values yl, y2, of a variable y (frequen- 
cy of 1 eyes on screen") at time tl, t2, thus y Is a function of t. 

A time series with variables y at specified times Is pictorlally 
represented In Figure 2, The data, expressed In this way, yields a graph 
of a time series of attention frequency of 5 children during a 5 minute 
film segment , 

After graphing attention data for several films, It was apparent 
that Identifiable similarities In the curves could be discriminated; 
however, the cause of these similarities (population variables or media 
variables) remained unknown. It was determined to use a moving average 
computation to plot the points on the graph In order to remove some 
errors of observation and to smooth out the curve. This was done by 
assigning a value to a point equal to the average of the preceding point 
(observed frequency), present frequency point, and following frequency 
poi nts. 

A study o* the utility of the moving average curve Is now In 
progress. The concerns and questions being answered are: 

I i With attention as the variable, what is the cause of 
Its variability? Is It In terms of content, mode of 
presentation, stimulus properties, captions, etc, 
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2. Can the attention curve be used to verify and predict 
attention for other films? 

3. Can strong general trends be Identified? That Is, to what 
extent Is what we observe responsible for variability of 
attention; and are there Inconsistencies In observations? 

In our Investigation so far, we find a graphic representation of 
attention data (attention profile) useful In several ways: 
1. It Is used as a check on priority classification of 

films. That is, if a film has a high attention profile, It 
confirms the classification of a film as being good. 
2* The attention profile Is used to determine attention spans 
and viewing characteristics of a given classroom, e.g., dif- 
ferences between the curves of viewers have been found to 
correspond to developmental variables such as age* reading 
achievement, etc. 
3. Another use Is (n pointing out common attention areas of 

a specific media across varlojs groups. 
CBP findings on attention seem to Indicate that different present 
atlons have different profiles; that younger (6 - 10 years, EHH) 
children have a greater range of variance than older children; that a 
relationship exists between high mean attention and content learned; 
and that media can be grouped and ranked on attention variables. 

Many unanswered questions persist; however, at present, the 
attention procedure adds dimensions to cur research. The procedures 



used to obtain the attention profile da^a are currently being reviewed 
In terms of their relevance In evaluating f 1 1ms > Although a film may 

: \ 

maintain attention, other effects In termk of subsequent behavior by 

the student have not been measured. However, efforts In developing 

reliable procedures for maintaining attention during film presentations 

are continuing based on results obtained from the procedures reported 
here, 
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